The present invention relates to processor structure in general and to multithreading processors in particular.
Multithreading is a process of breaking a program up into threads where one thread can be executed while another thread is being executed. Multithreading differs from parallel processing in that multithreading can be done with a single processor. If a processor were always active, performing an operation in each processor clock cycle, multiprocessing would not improve the processor's performance, because the total number of instructions performed by the processor is usually not reduced by multiprocessing. However, where one section of code might stall a processor, another section of code could be processed during that stall period where the processor would otherwise be idle. For example, if multiple clock cycles were needed to retrieve memory following a cache miss, a single-threaded processor would just wait for the results from the slow memory. However, with multithreading, a second thread could be started when the first thread stalls.
Ideally, the second thread would be independent of the first thread. Being independent means that the processing done in the second thread does not depend on a memory value or register value that is to be changed by the first thread and the processing done in the first thread does not depend on a memory or register value that has already been changed by the second thread. With this independence, the two threads can be executed at the same time without violating the causality normally expected in a program, i.e., that the effects of later instructions (the second thread) are not experienced by earlier instructions (the first thread) and the intended effect of the earlier instructions on the later instructions still occurs even if some of the instructions in the second thread are executed before some instructions in the first thread.
Speculation is another process by which processor performance can be improved. Speculation is often done with pipelined processors, where a processor speculates on the outcome of an operation that is done by a later stage in the pipeline in order to keep earlier stages busy and filling the pipeline. One common use of speculation is for branch prediction. If an early stage of a pipeline processes instructions and a later stage determines which of two branches to take, the pipeline might speculate on which branch will end up being taken, so that the pipeline can process instructions in hopes of that speculation being correct. Of course, for speculation to be worthwhile, it must save enough processing time and be right enough of the time to overcome the extra effort required for speculation. Speculating requires extra effort in the form of processing to make the speculation as well as processing to ensure that the effects of a speculation can be reversed if that speculation later turns out to be wrong.